Project & Program Evaluation
When you set out to do something in a community, you want to know what you’ve done, how it worked, what you can improve for the future, and what impact it’s had. This guide helps outline the basics of project & program evaluation.
Download Blue Page: Project & Program Evaluation
Additional Resources:
Georgia Council for the Arts Evaluation Guide
Createequity: Public Art and the Challenge of Evaluation
WHY EVALUATE?
Evaluation informs stakeholders about the outcomes and impacts of a given project.
It is critical for art, design, and cultural projects to be evaluated for a variety of reasons including:
- Acknowledging lessons learned
- Improving projects and programs
- Knowing what you’ve done and its impact
- Providing funders with proof of impact of initiatives
- Demonstrating impact to change policies
- Participating in scholarly discourse about practice through critical evaluation and consideration
TAKE STEPS TOWARDS EVALUATION
The following might be the standard steps of evaluation, but they may not work for all types of creative projects. It’s best to plan for evaluation while planning your project so you can capture critical insights and data along the way, not just at the end.
1. Understand resources required: What support will you need from: students, community partners, and the Office for Socially Engaged Practice? What time can you dedicate to this assessment? Do you need additional funding to execute?
2. Define your stakeholders: Who are the people involved? Involve them in clarifying objectives and providing feedback.
3. Describe the project or program: Articulate what your project or program is, what you want to accomplish, and what activities you’ll take on to pursue these goals.
4. Focus the design of your evaluation: An evaluation can consider process, means, resources, outcomes, activities, or outputs. Define the question you are trying to answer, how the information will be used, and the methods that are best suited to collecting this information.
5. Gather evidence: Using qualitative or quantitative methods, collect evidence that helps answer these questions.
6. Draw conclusions: Using the data you’ve collected, analyze based on the questions you defined. You may need to make comparisons to show where you sit relative to targets or other programs.
7. Disseminate findings & develop recommendations: Finally, share your conclusions with others on your team and those who can grow from your critical self-assessment.
Separate Project Evaluation from Learning Assessment
Project and program evaluations separately assess the content, work, and/or process through the outcome and impact in the community.
Student learning and faculty performance, while essential to engaged teaching, are separately evaluated. Student learning is assessed through in-class assignments and grades, while faculty performance is assessed through course evaluations.
EVALUATING CREATIVE PROJECTS
Evaluating a creative project may be complex, but that’s no reason not to try. The results can be powerful indicators of the value of your teaching or practice.
Challenges for creative projects
- Many community-based creative activities intentionally challenge the idea of a single, pre-defined idea of value, which is frequently not reflected in evaluation frameworks or processes.
- Cultural activities are often complex and unfold in non-linear, unpredictable ways, posing challenges for evaluation, particularly where resources are limited.
- Partnerships between arts and non-arts agencies can mean differing assumptions about success and measurement.
For creative projects, consider:
- Who participated in the process and what was that like for them?
- What are the new tangible skills learned by those who participated?
- What was the response to the work created (in person, online, etc.)?
- What were the goals of the process and the outputs? Were they accomplished?
- What resources did you use and how did you get access to them?
CONSIDER THE VALUES OF YOUR PROJECT OR PROGRAM
A framework of values may help you organize your thoughts about what you want to evaluate and assess.
Artistic Values: visual/aesthetic enjoyment, design quality, social activation, innovation/risk, host participation, challenge/critical debate.
Social Values: community development, poverty and social inclusion, health and well being, crime and safety, interpersonal development, travel/access, and skills acquisition.
Environmental Values: vegetation and wildlife, physical environment improvement, conservation, pollution and waste management, air, water and ground quality, and climate change and energy.
Economic Values: marketing/place identity, regeneration, tourism, economic investment and output, resource use and recycling, education, employment, project management/sustainability, and value for money.
Framework derived from Ixia’s “Public Art: A Guide to Evaluation.”
RESEARCH METHODS
Research methods could be quantitative (numbers), qualitative (description), or mixed (both).
Quantitative Methods:
- Counting (how many people attended, how long did it last, how many flyers were taken)
- Measure (steps taken, biometric responses, distance covered, area utilized)
Qualitative Methods:
- Surveys
- Interviews
- Observation
- Pre/Post Tests (measuring beginning and end of project/program)
- SWOT (Strengths, Weaknesses, Opportunities, Threats) Analysis
- ORID (Objective, Reflective, Interpretive, Decisional) Analysis
RESOURCES FOR EVALUATION
No matter what, you don’t have to go at evaluation alone.
Annual Collaborator Survey
Each year, the Office for Socially Engaged Practice surveys collaborators who worked with courses, research initiatives, or other programs in the Sam Fox School to gather their feedback on the relationship. This survey typically goes out immediately following commencement.
Evaluation Consultations
We can help you design a custom evaluation approach for your course or project based on your needs and how you plan to use the data. Contact the Office for more information.
Internal Review Board Support
Depending on your project and evaluation methods, you may need approval from the Internal Review Board (IRB) for your evaluation methods. If you seek to publish the results of your evaluation, this is necessary. Visit hrpo.wustl.edu.
REFERENCES
These resources may be useful as you develop methods for evaluating your own projects.
The Georgia Council for the Arts Evaluation Guide also provides an introduction to the different elements of project evaluation in the arts.
The Createequity webpage has a list of potential indicators of success that could be used for arts project evaluations.